Improved Video Coding through Texture Analysis and Synthesis

A new video coding approach based on texture analysis and synthesis is presented. The underlying assumption of our approach is that the textures in a video scene can be labeled subjectively relevant or irrelevant. Relevant textures are defined as containing subjectively meaningful details, while irrelevant textures can be seen as image content with less important subjective details. We apply this idea to video coding using a texture analyzer and a texture synthesizer. The texture analyzer (encoder side) identifies the texture regions with unimportant subjective details and generates side information for the texture synthesizer (decoder side), which in turn inserts synthetic textures at the specified locations. In this paper, it is shown that bit-rate savings of up to 19.4% can be achieved, using a semi-automatic MPEG-7-aided texture analyzer, compared to a standard conforming H.264/AVC video codec. The subjective video quality is thereby comparable to the H.264/AVC codec without the presented approach.